Measuring Engagement in Robot-Assisted Autism Therapy: A Cross-Cultural Study

During occupational therapy for children with autism, it is often necessary to elicit and maintain engagement for the children to benefit from the session. Recently, social robots have been used for this; however, existing robots lack the ability to autonomously recognize the children’s level of engagement, which is necessary when choosing an optimal interaction strategy. Progress in automated engagement reading has been impeded in part due to a lack of studies on child-robot engagement in autism therapy. While it is well known that there are large individual differences in autism, little is known about how these vary across cultures. To this end, we analyzed the engagement of children (age 3–13) from two different cultural backgrounds: Asia (Japan, n = 17) and Eastern Europe (Serbia, n = 19). The children participated in a 25 min therapy session during which we studied the relationship between the children’s behavioral engagement (task-driven) and different facets of affective engagement (valence and arousal). Although our results indicate that there are statistically significant differences in engagement displays in the two groups, it is difficult to make any causal claims about these differences due to the large variation in age and behavioral severity of the children in the study. However, our exploratory analysis reveals important associations between target engagement and perceived levels of valence and arousal, indicating that these can be used as a proxy for the children’s engagement during the therapy. We provide suggestions on how this can be leveraged to optimize social robots for autism therapy, while taking into account cultural differences.MEXT Grant-in-Aid for Young Scientists B (grant no. 16763279)Chubu University Grant I (grant no. 27IS04I (Japan))European Union. HORIZON 2020 (grant agreement no. 701236 (ENGAGEME))European Commission. Framework Programme for Research and Innovation. Marie Sklodowska-Curie Actions (Individual Fellowship)European Commission. Framework Programme for Research and Innovation. Marie Sklodowska-Curie Actions (grant agreement no. 688835 (DE-ENIGMA)

Measuring Engagement in Robot-Assisted Autism Therapy: A Cross-Cultural Study

During occupational therapy for children with autism, it is often necessary to elicit and maintain engagement for the children to benefit from the session. Recently, social robots have been used for this; however, existing robots lack the ability to autonomously recognize the children’s level of engagement, which is necessary when choosing an optimal interaction strategy. Progress in automated engagement reading has been impeded in part due to a lack of studies on child-robot engagement in autism therapy. While it is well known that there are large individual differences in autism, little is known about how these vary across cultures. To this end, we analyzed the engagement of children (age 3–13) from two different cultural backgrounds: Asia (Japan, n = 17) and Eastern Europe (Serbia, n = 19). The children participated in a 25 min therapy session during which we studied the relationship between the children’s behavioral engagement (task-driven) and different facets of affective engagement (valence and arousal). Although our results indicate that there are statistically significant differences in engagement displays in the two groups, it is difficult to make any causal claims about these differences due to the large variation in age and behavioral severity of the children in the study. However, our exploratory analysis reveals important associations between target engagement and perceived levels of valence and arousal, indicating that these can be used as a proxy for the children’s engagement during the therapy. We provide suggestions on how this can be leveraged to optimize social robots for autism therapy, while taking into account cultural differences.MEXT Grant-in-Aid for Young Scientists B (grant no. 16763279)Chubu University Grant I (grant no. 27IS04I (Japan))European Union. HORIZON 2020 (grant agreement no. 701236 (ENGAGEME))European Commission. Framework Programme for Research and Innovation. Marie Sklodowska-Curie Actions (Individual Fellowship)European Commission. Framework Programme for Research and Innovation. Marie Sklodowska-Curie Actions (grant agreement no. 688835 (DE-ENIGMA)

Generating Robotic Speech Prosody for Human Robot Interaction: A Preliminary Study

The use of affective speech in robotic applications has increased in recent years, especially regarding the developments or studies of emotional prosody for a specific group of people. The current work proposes a prosody-based communication system that considers the limited parameters found in speech recognition for the elderly, for example. This work explored what types of voices were more effective for understanding presented information, and if the affects of robot voices reflected on the emotional states of listeners. By using functions of a small humanoid robot, two different experiments conducted to find out comprehension level and the affective reflection respectively. University students participated in both tests. The results showed that affective voices helped the users understand the information, as well as that they felt corresponding negative emotions in conversations with negative voices

Social Distance in Interactions between Children with Autism and Robots

The use of non-industrial robots, called service robots, is increasing in the welfare fields to meet the demand for robot therapy among individuals with autism. The more simple communication structures and repetitive behaviors of robots, compared to humans, make it easier for children with autism to interpret communication and respond appropriately. Interacting with a robot allows for social distance to be designed and maintained depending on a person’s social interaction needs. To simulate natural social interactions, robots need to perform social distance in some way. In the context of interacting with autistic children, understanding their social response levels is crucial for the robot to implement decisions regarding the distance kept during the interaction. In this study, an experiment was conducted to examine the accuracy of a detection program and explore the correlations between the social responsiveness of children and social distance, wherein 15 autistic children interacted with a robot on a one-to-one basis for about 20 min. The results revealed that both programs implemented in the robot-assisted autism therapy were effective in detecting social distance in a natural HRI situation

Understanding Emotions in Children with Developmental Disabilities during Robot Therapy Using EDA

Recent technological advancements have led to the emergence of supportive robotics to help children with developmental disabilities become independent. In conventional research, in robot therapy, experiments are often conducted by operating the robot out of the subject’s sight. In this paper, robot therapy using a system that can autonomously recognize the emotions of a child with developmental disabilities and provide feedback was developed. The aim was to quantitatively infer emotional changes in children using skin conductance (EDA) during robot therapy. It was demonstrated that the robot could recognize emotions autonomously and provide feedback to the subjects. Additionally, a quantitative evaluation was conducted using EDA. By analyzing the symptoms related to developmental disorders, it may be possible to improve the recognition rate and tailor therapy based on symptoms

Environment aware ADL recognition system based on decision tree and activity frame

The interest towards robots for elderly care has been growing in the last years. Systems aiming to integrate robot interactive components and the user’s activity recognition system are increasing as well. This work presents an activity aware intelligent system that supports user in his/her daily life tasks. The proposed system aims to integrate three important aspects into a smart house application (environment monitoring, user activity recognition and user friendly interaction). The information gathered from sensors across the environment is structured as the state of the environment in a compacted form called activity frame. This specific frame is used by a predictor (based on the decision tree method), in order to recognize the activities that have been performed by the user inside his/her domestic environment. The recognized activity is used by an user-interactive component, which uses the predicted behavior as a guideline for its interaction planner. The presented activity recognition system was tested with the data provided by different smart home projects, and the recognition rate for the proposed predictor has high recognition rate compared to other similar ones. The architecture described by the sensory network allows the system to be easily implemented in real time in a smart house context

Finding Characteristics of Users in Sensory Information: From Activities to Personality Traits

The main objective of this work was to use information provided by a sensor-based activity recognition system to create a profile comprising user habits and link this information to the personality traits of users. User habits are represented by the sequence and duration of often observed daily life activities. Based on this information, we represented the user trials (sequence of activities) following a numerical method using Fourier series. The duration and sequence of the activities changed the phase and amplitude of the harmonics present in the Fourier representation. Each trial represented in this manner is called a behavioral spectrum. These data and the scores obtained from personality questionnaires were clustered separately and then an association was created between the clusters. The objective was to associate the activity-related features (sensor-based) and personality traits. The experimental results showed that for both young and elderly subjects, there is an association between the user personality traits and the manner in which they perform their activities. Moreover, the results obtained in this work show a promising method of assessing the personality traits of users based on their activities

Enhancement of the communication efficiency of interactive robots for autism therapy by using touch and colour feedback

Previous studies in the field of robot assisted therapy demonstrated that robots engage autistic children’s attention in a better way. Therefore, the interactive robots appear to be a promising approach for improving the social interaction and communication skills of autistic children. However, most of the existing interactive robots use a very small number of communication variableswhich narrow their effectiveness to a few aspects of autistic childrens’ social communication behaviour. In the present work, we explore the effects of touching and colours on the communication effectiveness between a robot and an autistic child and their potential for further adjustability of the robot to child’s behaviour. Firstly, we investigated touching patterns of autistic and non-autistic children in three different situations and validated their responses by comparison of touching forces. Results showed that patterns of touching by non-autistic children have certain consistency, while reaction patterns in autistic children vary from person to person. Secondly, we studied the effect of colour feedback in autism therapy with the robot. Results showed that participants achieved better completion rate when colour feedback was provided. The results could support the design of more effective therapeutic robots for children with autism

Culturenet: a deep learning approach for engagement intensity estimation from face images of children with autism

© 2018 IEEE. Many children on autism spectrum have atypical behavioral expressions of engagement compared to their neu-rotypical peers. In this paper, we investigate the performance of deep learning models in the task of automated engagement estimation from face images of children with autism. Specifically, we use the video data of 30 children with different cultural backgrounds (Asia vs. Europe) recorded during a single session of a robot-assisted autism therapy. We perform a thorough evaluation of the proposed deep architectures for the target task, including within- and across-culture evaluations, as well as when using the child-independent and child-dependent settings. We also introduce a novel deep learning model, named CultureNet, which efficiently leverages the multi-cultural data when performing the adaptation of the proposed deep architecture to the target culture and child. We show that due to the highly heterogeneous nature of the image data of children with autism, the child-independent models lead to overall poor estimation of target engagement levels. On the other hand, when a small amount of data of target children is used to enhance the model learning, the estimation performance on the held-out data from those children increases significantly. This is the first time that the effects of individual and cultural differences in children with autism have empirically been studied in the context of deep learning performed directly from face images